The goal of processor verification is to ensure equivalence of a processor and its architectural specification. This goal can be achieved either by a formal proof or by exhaustive simulation. However, the complexity of processors renders the formal approach impractical for most industrial applications and the size of the test space makes exhaustive simulation impossible.
Typically, the architecture specification is an informal description of the processor's resources, the instruction repertoire and the effect of instruction execution on the processor state. It also describes the main hardware mechanisms such as address translation, interrupt handling or multi-tasking. Computer architectures are complex. A typical architecture includes hundreds of instructions; a few dozens of resources, such as main memory, general-purpose registers, special-purpose registers, and complex functional units, e.g. floating point, address translation, external interrupt mechanism. A typical architecture description is a few hundred pages long.
In practice, design verification of hardware processors is generally, but not exclusively, carried out by simulating the operation of sets of assembly level test programs using a hardware simulator with a particular set of design parameters and comparing the results with the output predicted by a behavioral simulator.
The hardware simulator may take as input a design model written in a hardware description language, such as the IEEE standard VHDL (IEEE standard 1076-1987). Of course, if prototype hardware is available the test programs may be run directly on that. The behavioral simulator is normally written for software development purposes prior to having the actual hardware and represents the expected behavior of the hardware given the architecture specification.
Traditionally, these test programs are written manually. First, a list of test requirements is compiled; then tests are written to cover this list. Requirements derived from the architecture specification usually call for the testing of every instruction, under normal, boundary and exception conditions. The tests written to satisfy these requirements are called Architecture Verification Programs (AVP), and are required to run correctly on any realization of the architecture. The design work-book, which defines major design details, such as clock cycle and cache size, and the actual HDL design are used to construct a second list of requirements and the Implementation Verification Programs (IVP). The latter typically test the functionality of caches, pipelines and units like a Carry Look Ahead adder or a bit-rotator.
Design verification by manually written tests is not cost effective. Using this approach a substantial proportion of the processor design effort must be dedicated to verification. Furthermore, many of these tests are too simple, as test engineers find it difficult to define complex situations.
The automatic generation of such test programs for the verification of processor architectures is known from A. Aharon, et al. `Verification of the IBM RISC System/6000 By a Dynamic Biased Pseudo-Random Test Program Generator`, IBM Systems Journal, April 1991 referred to hereinafter as R11 and from EP-A-453394. The automation of test program generation has increased productivity and in recent years has also provided better quality.
Typically, such a test program generator is a complex software system. An example described in R1 written in the programming language `C` spans about 150,000 lines of code. The complexity of processor architectures, which nowadays can include hundreds of instructions and around ten functional units, and their informal description are reflected in the complexity of the test generation system.
These known test program generators suffer from the drawback that a new test program generator has to be developed and implemented for each architecture for which the process is to be used. In other words, the generator is architecture dependent.
Furthermore, changes in the architecture or in the testing requirements require subtle modifications to be made to the generator's code. Since design verification gets under way when the architecture is still evolving, a typical test generation system undergoes frequent changes.
Typically, there are two levels of changes: within an architecture version and between versions. Within a version there are normally hundreds of changes, many of them subtle. When the architecture is relatively stable, an architecture version is declared. It may be substantially different from the previous version. The changes in both levels are described informally, are often difficult to identify, and their consequences for the test program generator are not always clear. Furthermore, many of the changes required in the test generator are due to the evolving testing knowledge accumulated through the validation process itself. New testing needs rise frequently as a consequence of previous testing results and uncovering of design errors.
In the prior art, features of the architecture and knowledge gained from testing are modelled in the generation system. The architecture is needed to generate legal instructions and tests. Testing knowledge is used to generate interesting, smart or revealing instructions and tests. This knowledge is embedded in the generation procedures of the prior art systems. Modelling of both architecture and testing knowledge is procedural and tightly interconnected. Its visibility is thus low. This worsens the effects of the complexity and changeability.